For this analysis we use the Cleveland "Coronary Artery Disease" dataset found on the UCI Machine Learning Repository
at the following location:
Heart Disease Dataset
The objective of the analysis is to use statistical learning to identify factors associated with Coronary Artery
Disease as indicated by a coronary angiography interpreted by a Cardiologist.
According to the paper by (Detrano et al., 1989) the data represents data collected for 303 patients referred for
coronary angiography at the Cleveland Clinic between May 1981 and September 1984. The 13 independent/ features
variables can be divided into 3 groups as follows:
Routine evaluation (based on historical data):
-
ECG at rest
-
Serum Cholesterol
-
Fasting blood sugar
Non-invasive test data (informed consent obtained for data as part of research protocol):
-
Exercise ECG
-
ST-segment peak slope (upsloping, flat or downsloping)
-
ST-segment depression
-
Excercise Thallium scintigraphy (fixed, reversible or none)
-
Cardiac fluoroscopy (number of vessels appeared to contain calcium)
Other demographic and clinical variables (based on routine data):
-
Age
-
Sex
-
Chest pain type
-
Systolic blood pressure
-
ST-T-wave abnormality (T-wave abnormality)
-
Probably or definite ventricular hypertrophy (Este's criteria)
The dependent/ response variable was the angiographic test result indicating a >50% diameter narrowing.
Detrano et al created 352 logistic regression models based on different combinations of the previously discussed
13 variables. Depending on information available in the test dataset one of these models was applied during model
validation.
Unfortunately the the paper by (Detrano et al., 1989) does not publish extensive model accuracy results such as:
overall accuracy, C-Statistic, precision (positive predictive value), recall (sensitivity) or specificity. The
model was however validated on various external datasets which is very important. The positive predictive value
(precision) is reported for a range of probability cut-off points for the various test sets. The various data sets
have widely varying prevalence (38% - 81%), and hence te positive predictive value likewise varies widely between the
reported tests.
From these results one can see that the model was capable of a Precision of approximately 75% for datasets closely
matching the Cleveland dataset in terms of disease prevalence within the test set.
We are going to replicate the analysis by performing a logistic regression and will perform a validation study to
confirm results obtained by (Detrano et al., 1989) in terms of model accuracy. Furthermore we will do an analysis of
significance of the various logistic regression features as means of classifying feature importance.
Subsequent to this analysis we will use an ensemble model (Random Forest) to create feature importance classifications
to validate the regression findings. We will also apply our learnings to a Neural Network, to assess performance on
the smaller dataset i.e. after features selection has been applied.
We aim to achieve this by following the Machine Learning pipeline approach of deploying a variety of ML techniques to
build a predictive model and analyse its results. In the process we hope to gain valuable insights. The various steps
in the process are as follows (not necessarily in this order):
-
Load data
-
Prepare data
-
Clean data
-
Missing values
-
Outliers
-
Erroneous values
-
Explore data
-
Exploratory descriptive analysis (EDA)
-
Correlation analysis
-
Variable cluster analysis
-
Transform Data
-
Engineer features
-
Encode data
-
Scale & normalise data
-
Impute data
-
Feature selection/ importance analysis
-
Build model
-
Model selection
-
Data sampling (validation strategy, imbalanced classification)
-
Hyperparameter optimisation
-
Validate model
-
Analysis of results
-
Response curves
-
Accuracy analysis
-
Commentary
Let us start the analysis!
